Spatial Interactions between Humans and Assistive Agents

While computers assist humans with tasks such as navigation that involve spatial aspects, agents that can interact in a meaningful way in this context are still in their infancy. One core issue is the mismatch in the representation of spatial information a computer-based system is likely to use, and the one a human is likely to use. Computers are better suited for quantitative schemes such as maps or diagrams that rely on measurable distances between entities. Humans frequently use higher-level, domain-specific conceptual representations such as buildings, rooms, or streets for orientation purposes. Combined with the person-centric world view that we often assume when we refer to spatial information, it is challenging for agents to convert statements using spatial references into assertions that match their own internal representation. In this paper, we discuss an approach that uses natural language processing and information extraction tool kits to identify entities and statements about their spatial relations. These extractions are then processed by a spatial reasoner to convert them from the human conceptual space into the quantitative space used by the computer-based agent. Motivation and Background This paper investigates the domain of human-agent interaction in spatial contexts. As an example, we use a scenario where a person explores an exhibit at a museum, supported by an agent that offers guidance about the layout and contents of the exhibit. Our emphasis here is on the spatial structure of the museum, and the arrangement of the pieces on display, not on information about the pieces themselves. We also make the assumption that human and agent communicate in a possibly restricted natural language in written or spoken form, or a mixture of both. For the sake of simplicity, we assume that the input to the agent is provided as text, so as to avoid the complications of speech recognition. The core issues of our scenario are • communication about spatial aspects of the environment • spatial aspects of co-existence in environments ∗The author gratefully acknowledges funding by the German Research Foundation (DFG) by way of the SFB/TR 8 Spatial Cognition Project [DesignSpace]. Copyright c © 2011, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Mehul Bhatt∗ SFB/TR 8 Spatial Cognition University of Bremen, Germany • planning and coordination of activities with spatial constraints In the simplest case, an instance of the agent could be a smart phone with sensors that convey visual information (e.g. a camera) and localization information (such as GPS or a WiFi-based system), combined with a map of the building and rooms that host the exhibit. In this case, the mobility of the agent is dependent on the human, and the operation of the camera also requires human assistance. A more sophisticated agent could be a robot that acts as a “tour guide” for the human. This agent has a significantly higher degree of autonomy, and also a more capable set of sensors. In either case, the agent can guide the human through the exhibit on a pre-defined or customized tour, but also respond to queries about the exhibit and the layout of the building (e.g. “Where are the bathrooms?”, “What’s the quickest way to the Mona Lisa?”). in the following, we examine the interaction between humans and agents in this scenario, with particular emphasis on spatial aspects. The work described here is under partial implementation in research projects on information extraction (Assal et al. 2010) and spatial cognition within assistance systems (Bhatt 2010b; Flanagan 2010; Bhatt and Guesgen 2009; Guesgen and Bhatt 2010; Bhatt and Freksa 2010). The work discussed here combines aspects from two areas: Interactions between agents, in particular communication between computer-based assistive agents and humans, and dealing with spatial aspects of the environment in which the agents operate. In this section, we will examine related work in a few fields most relevant for our overall purpose. Human-Agent Interaction When humans interact with computer-based systems, the current interaction methods are typically constrained by the capabilities and limitations of computer systems, and the onus is on the human counterpart to adapt to those constraints. This is acceptable under many circumstances, especially if computers are used as tools to support specific tasks such as viewing or writing documents, or organizing information in a data base or spreadsheet. Considering the physical interaction space in which these exchanges take place, the human and computer counterparts are typically within a distance that allows humans to use their hands to interact, and to view items displayed on a screen com-